feat: add ASOF join logical semantics - #23829

## Which issue does this PR close?

- Part of #318.
- Umbrella PR: #23738.
- Builds on #23829 and #23828, both merged.

## Rationale for this change

This is the SQL frontend layer of the ASOF JOIN stack. It adds syntax
and
unparsing on top of the merged physical and logical contracts. The PR
now
contains only the isolated SQL frontend diff.

## What changes are included in this PR?

- Plan `ASOF JOIN ... MATCH_CONDITION (...)` with optional `ON` or
`USING`
  equality keys.
- Reject unsupported match shapes and non-equality `ON` predicates.
- Unparse ASOF joins while preserving right-side candidate preselection
and
  nested join scope.
- Document the supported SQL syntax and semantics, including qualified
`USING` keys, left-partitioned broadcast execution, the full-right
memory
requirement, repeated scans, and the absence of spill/repartitioned
ASOF.
- Add SQL integration and sqllogictest coverage for all four match
directions,
coercion, equality-free joins, USING, invalid contracts, EXPLAIN,
boundedness,
  and optimized-plan round trips.
- Verify the broadcast topology with a multi-partition left input: left
  partitioning is preserved while the right input is single-partitioned.

## Are these changes tested?

Yes:

- `cargo fmt --all`
- `./ci/scripts/doc_prettier_check.sh --write --allow-dirty`
- `cargo clippy --all-targets --all-features -- -D warnings`
- `cargo test -p datafusion --test core_integration asof --all-features`
- `cargo test -p datafusion-sqllogictest --test sqllogictests
--all-features -- asof_join`
- The extended workspace test command from the contributor guide

## Are there any user-facing changes?

Users can express Snowflake-style ASOF joins in SQL with
`MATCH_CONDITION`, optional equality keys, and `<`, `<=`, `>`, or `>=`
match
directions. With `USING`, wildcard output exposes one unqualified key
while
both qualified input keys remain addressable. The user guide also
documents
the initial broadcast strategy and its memory/no-spill limitations.

#23829 and #23828 are merged. This is the next core layer in the ASOF
stack and
does not depend on the optional floating-point follow-up #24375.

* feat: add ASOF join physical operator (apache#23828)

## Which issue does this PR close?

- Part of apache#318.
- Umbrella PR: apache#23738.
- Follow-up for floating-point equality keys: apache#24375.

## Rationale for this change

This is the first layer of the ASOF JOIN stack. It establishes a
broadcast-based physical execution contract independently so later
floating-point equality, logical-plan, SQL, DataFrame, and serialization
changes
can be reviewed as smaller follow-up PRs.

The initial implementation deliberately favors the simpler broadcast
design:
the right input must fit in memory and each left partition scans the
shared
right-side batches. A repartitioned implementation can be evaluated
separately
without changing the ASOF semantics introduced here.

Floating-point equality keys are rejected in this base layer because
Arrow's
required sort order distinguishes `-0.0` from `+0.0` while join equality
does
not. apache#24375 adds the required ordering normalization as an independently
reviewable layer.

## What changes are included in this PR?

- Add `AsOfJoinExec` for left-preserving, Snowflake-style ASOF
semantics.
- Coalesce and collect the ordered right input once, then share it
across all
  left partitions.
- Keep the left input partitioned so each partition can scan
independently and
  preserve the left-side output partitioning.
- Preserve merge state across input and output batch boundaries.
- Reserve each retained Arrow buffer exactly once, including when
right-side
batches are zero-copy slices, and expose build, match, and output
metrics.
- Define output properties and statistics for the broadcast execution
model.
- Reject floating-point equality keys until apache#24375 supplies a
sort/equality
  contract that handles signed zero correctly.
- Add physical operator tests covering match directions, equality
groups,
batch boundaries, unmatched rows, invalid contracts, shared-buffer
memory
accounting, multi-partition broadcast execution, and float-key
rejection.

## Are these changes tested?

Yes:

- `cargo fmt --all`
- `cargo clippy --all-targets --all-features -- -D warnings`
- `cargo test -p datafusion-physical-plan joins::asof_join
--all-features`
- Extended workspace tests from the contributor guide
- FFI integration tests

## Are there any user-facing changes?

This adds a new physical operator API. The base operator deliberately
rejects
floating-point equality keys; apache#24375 adds full Float16, Float32, and
Float64
support. SQL and DataFrame APIs are left to later dependent PRs.

---------

Co-authored-by: Yongting You <2010youy01@gmail.com>

* feat: add ASOF join logical semantics (apache#23829)

- Part of apache#318.
- Umbrella PR: apache#23738.
- Depends on apache#23828 (merged).

This is the logical-planning layer of the ASOF JOIN stack. It defines
the
logical contract and planner behavior separately from the SQL frontend
and
serialization formats.

logical
layer. It no longer depends on the optional floating-point follow-up

- Add `LogicalPlan::AsOfJoin`, `AsOfJoin`, and `AsOfMatch`.
- Validate deterministic expressions, input ownership, supported match
  operators, equality-key types, and USING constraints.
- Add `LogicalPlanBuilder` entry points and schema construction that
preserves
both qualified `USING` keys while exposing one unqualified wildcard key.
- Integrate ASOF joins with tree transforms, display, type coercion,
projection
  pruning, row bounds, and physical planning.
- Plan logical ASOF joins to the broadcast-based `AsOfJoinExec` from
- Fail closed at proto, SQL unparser, and Substrait boundaries until
their
  owning stack layers add explicit support.
- Defer ASOF-specific functional-dependency refinement to apache#24799 and
filter
  pushdown to apache#24801 so each optimization can be reviewed independently.

Yes:

- `cargo fmt --all`
- `cargo clippy --all-targets --all-features -- -D warnings`
- `cargo test -p datafusion-expr min_rows_of_joins --all-features`
- `cargo test -p datafusion-substrait
asof_join_fails_closed_until_substrait_has_an_extension --all-features`
- The extended workspace test command from the contributor guide

This adds logical-plan and builder APIs for ASOF joins. SQL syntax,
DataFrame
APIs, and plan serialization are intentionally left to dependent stack
PRs.
Floating equality keys remain rejected by the merged physical operator
unless
the independent follow-up apache#24375 is also included.

As with any new public `LogicalPlan` variant, downstream exhaustive
matches must
add an arm. The variant is appended so existing variants retain their
`PartialOrd` ordering; maintainers should still treat the enum addition
as a
Rust source-compatibility break.

This PR can be reviewed independently now that apache#23828 has merged. The
optimization follow-ups apache#24799 and apache#24801 are not required by the core
ASOF
stack.

* feat: support ASOF JOIN SQL (apache#23830)

- Part of apache#318.
- Umbrella PR: apache#23738.
- Builds on apache#23829 and apache#23828, both merged.

This is the SQL frontend layer of the ASOF JOIN stack. It adds syntax
and
unparsing on top of the merged physical and logical contracts. The PR
now
contains only the isolated SQL frontend diff.

- Plan `ASOF JOIN ... MATCH_CONDITION (...)` with optional `ON` or
`USING`
  equality keys.
- Reject unsupported match shapes and non-equality `ON` predicates.
- Unparse ASOF joins while preserving right-side candidate preselection
and
  nested join scope.
- Document the supported SQL syntax and semantics, including qualified
`USING` keys, left-partitioned broadcast execution, the full-right
memory
requirement, repeated scans, and the absence of spill/repartitioned
ASOF.
- Add SQL integration and sqllogictest coverage for all four match
directions,
coercion, equality-free joins, USING, invalid contracts, EXPLAIN,
boundedness,
  and optimized-plan round trips.
- Verify the broadcast topology with a multi-partition left input: left
  partitioning is preserved while the right input is single-partitioned.

Yes:

- `cargo fmt --all`
- `./ci/scripts/doc_prettier_check.sh --write --allow-dirty`
- `cargo clippy --all-targets --all-features -- -D warnings`
- `cargo test -p datafusion --test core_integration asof --all-features`
- `cargo test -p datafusion-sqllogictest --test sqllogictests
--all-features -- asof_join`
- The extended workspace test command from the contributor guide

Users can express Snowflake-style ASOF joins in SQL with
`MATCH_CONDITION`, optional equality keys, and `<`, `<=`, `>`, or `>=`
match
directions. With `USING`, wildcard output exposes one unqualified key
while
both qualified input keys remain addressable. The user guide also
documents
the initial broadcast strategy and its memory/no-spill limitations.

stack and
does not depend on the optional floating-point follow-up apache#24375.

---------

Co-authored-by: Xuanwo <github@xuanwo.io>
Co-authored-by: Yongting You <2010youy01@gmail.com>

## Which issue does this PR close?

- Part of apache#318.
- Umbrella PR: apache#23738.
- Follows apache#23829 (merged).

## Rationale for this change

This is the serialization layer of the ASOF JOIN stack. It gives logical
and
physical ASOF plans explicit protobuf representations without coupling
wire
format review to SQL or DataFrame APIs.

apache#23829 and its prerequisites are merged. After restacking onto the
current
`main`, this PR now contains only the serialization layer.

## What changes are included in this PR?

- Add protobuf messages and enum values for logical and physical ASOF
joins.
- Encode and decode equality keys, ordered match expressions, match
direction,
  join constraint, and right output indices.
- Implement physical serialization through `ExecutionPlan::try_to_proto`
and
  `AsOfJoinExec::try_from_proto`, following the current self-serializing
  execution-plan pattern.
- Regenerate `prost` and `pbjson` sources with the repository generator.
The
  physical-plan oneof uses the next append-only tag after
  `PiecewiseMergeJoinExec`.
- Add logical round trips for all four match directions and a physical
  `AsOfJoinExec` round trip.

## Are these changes tested?

Yes:

- `./datafusion/proto-models/regen.sh`
- `cargo fmt --all`
- `cargo clippy --all-targets --all-features -- -D warnings`
- `./dev/rust_lint.sh`
- `cargo test -p datafusion-proto roundtrip_asof_join --all-features`
- The extended workspace test command from the contributor guide

## Are there any user-facing changes?

Logical and physical ASOF join plans can be serialized through
`datafusion-proto`. The additions use new messages and append-only
oneof/enum
tags, so existing wire tags are not reused. Generated public Rust enums
gain new
variants, however, so downstream exhaustive matches must add arms; this
is a
Rust source-compatibility break even though the wire additions are
compatible.

This PR can now be reviewed independently. It does not depend on the
optional
floating-point follow-up apache#24375.